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Abstract 



This paper uses data from the 1990 National Assessment of Educational Progress 
(NAEP) Trial State Assessment to describe educational opportunities for students in eight- 
grade mathematics in 1990. It presents preliminary analysis of the distribution of these 
opportunities across the state. Specifically, the paper addresses two questions: 

• What kinds of instructional practices and curricular emphases were used in different 
states to teacher eighth-grade math in 1990? How did these vary across states. 

• How and to what extent were differences in instructional practice and curricular 
emphases related to state characteristics and policies, such as level of spending, 
demographics and socio-economic characteristics, and state curriculum and testing 
mandates. 

The first section of the paper summarizes the methodology used in the 1990 NAEP 
Trial State Assessment, and in the analysis reported here. The second section contains 
findings of the study. The final section discusses implications of the findings for national 
educational policy and highlights a follow-up CPRE study using data from the 1992 
NAEP Trial State Assessment. 

The study found that a majority of eighth-grade students receive math instruction in 
traditional classrooms. They are assigned to classrooms based on their ability, they receive 
between 2.5 and 4 hours of mathematics instruction by teachers who place heavy emphasis 
on numbers and operation, facts and concepts. But teachers pay limited attention to 
geometry and statistics and probability. The students are more likely to do problems from 
textbooks than to do reports or problems on math, and few students regularly use 
calculators or computers. 

At the state level, however, instructional practices fall into two groups: "traditional" 
(heavy reliance on textbooks and teaching of numbers and operations) and "non- 
traditional" (less reliance on textbooks, less emphasis on numbers, more use of manipu- 
lates, small groups, reports, calculators and computers). 

Further, it appears that eighth-grade math teachers that have participated in at least 
16-hours of in-service training in math or the teaching of math in the last year are more 
likely to report using non-traditional practices. Although these patterns are based on 
aggregated data and simple statistical methods, this is the first study to look at relation- 
ships among instructional practices. These relationships are worth exploring further using a 
nation, rather than state-level, database, and at the fourth, as well as eighth-grade level. 
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Introduction 



Since the mid-1980s, state and local policymakers have taken steps to strengthen 
elementary and secondary curriculum, particularly in the areas of mathematics and science. 
State legislators and boards of education have increased coursework requirements for high 
school graduation, developed curriculum standards for K-12 education focusing on 
conceptual understanding rather than basic skills, sought greater rigor in textbooks and 
student outcomes, and aligned statewide assessment programs with the more ambitious 
curriculum standards. Professional organizations such as the National Council of Teachers 
of Mathematics established standards for a sound curriculum in mathematics (NCTM 
1989). We know little, however, about the impact of these activities on curriculum and 
instructional practices in the classroom, or on student performance. 

The push for higher standards, and student accountability tied to these standards, also 
raises an age-old concern for fairness and equity. Is it fair to tie decisions about high 
school graduation and entry to either the job market or post-secondary education to more 
rigorous tests if students don't have an opportunity to learn the new knowledge and skills 
that are being assessed? Do students have equal access to these opportunities or are they 
allocated differentially by student race/ethnicity, socio-economic status and/or community? 
The equity issue has also been advanced by state courts, which are beginning to define 
educational equity in terms of educational outcomes, rather than educational inputs, such 
as education expenditures per pupil. For example, the courts in Alabama. Kentucky and 
Massachusetts have defined the state responsibility for education to include opportunities 
for students to attain sufficient skills in different academic areas to function at a state, 
national and international level, to be informed citizens, to appreciate their own, and 
others 1 cultural heritage, to compete favorably with students in other states, and to live up 
to their full potential 

This concern for equity has led to a spirited public debate over the concepts of school 
delivery standards, or opportunity to learn standards. Should schools, school districts and/ 
or states prove they have provided adequate and equal access to appropriate educational 
opportunities before they can hold students, teachers and schools accountable for student 
performance? Porter (1993) argues that school delivery/opportunity to learn standards 
could be used for three purposes: (1) presenting a vision of good practice for educators to 
aspire to; (2) providing a framework for an indicator system that describes the extent to 
which schools have implemented good practice; and (3) providing the basis for school 
accountability. 

There is no consensus, however, on what should be included under the heading of 
school delivery standards or opportunity to learn standards. Many use the two terms 
interchangeably. Others distinguish the two. Porter (1993), for example, views opportunity 
to learn standards as a subset of school delivery standards. School delivery standards 
include such organizational features as school leadership, school goals, parent and 
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community support, and district and state support. Opportunity to learn focuses more r 
narrowly on "the enacted curriculum as experienced by the student" (Porter 1993, p. 7), 
features of the educational system that best predict student achievement. While Porter and 
others (1993) have studied the enacted curriculum in high school mathematics and science 
in a small sample of high schools, little data have been reported on either school delivery 
standards or opportunity to learn standards across schools and states. 

This paper uses data from the 1990 NAEP Trial State Assessment to describe 
educational opportunities for students in eighth-grade mathematics in 1990, and to present 
preliminary analyses of the distribution of these opportunities across states. Specifically, 
the paper address two questions: 

• What kinds of instructional practices and curricular emphases were used in 
different states to teach eighth -grade mathematics in 1990? How, and to what 
extent, did these practices and emphases vary across states? 

• How, and to what extent, were differences in instructional practices and 
curricular emphases related to variations in state characteristics and policies, such 
as level of education spending, the demographic and socio-economic 
characteristics of states and state curriculum policies and state testing mandates? 

While the variables included in the NAEP database do not provide the level of detail on 
curriculum and instruction reported by Porter and others (1993), they provide one set of 
broad indicators of the extent to which mathematics education reforms are being adopted 
across the country. 1990 NAEP data also provide a benchmark for tracking reform over 
time and for relating reform to student achievement. 

This paper is divided into three sections. The first section summarizes the method- 
ology used in the 1990 NAEP Trial State Assessment and in the analysis reported here. 
The second section contains the findings of the study. Section three discusses implications 
of the findings for national education policy and outlines a follow-up study using data 
from the 1992 NAEP Trial State Assessment. 
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Study Methodology 



Since 1969, the National Assessment of Educational Progress (NAEP) has served as 
The Nation's Report Card, regularly collecting and reporting information on the academic 
knowledge, skills, and attitudes of a national sample of 9-, 13-, and 17-year-olds in a 
variety of subject areas. Each assessment covers several subjects, many of which are 
reassessed periodically to monitor trends in achievement. In 1984, NAEP began sampling 
students in grades 4, 8 and 12, as well as by age. NAEP undertook a voluntary Trial State 
Assessment in 1990, assessing eighth-grade mathematics in 37 states, two territories and 
the District of Columbia. 1 The assessment and questionnaire instruments were identical to 
those used in the 1990 NAEP national assessment of mathematics. 

The 1990 NAEP mathematics assessment incorporated a new mathematics objectives 
framework and new assessment questions, reflecting the Curriculum and Evaluation 
Standards for School Mathematics, developed by the National Council of Teachers of 
Mathematics (NCTM 1989). The assessment included a broad range of questions that 
required students to use scientific calculators, provide responses using protractor/rulers, 
and solve problems in a constructed-response format. It tested five content areas (numbers 
and operations; measurement; geometry; data analysis, statistics and probability; and 
algebra and functions) and three mathematical abilities (conceptual understanding; 
procedural knowledge; and problem-solving). In addition, students, their teachers, and 
school administrators were asked to complete questionnaires about their background and 
instruction in mathematics. These questionnaires were designed to provide an educational 
context for understanding information on student achievement, to identify differences in 
access to instruction and distribution of services for various types of students, and to track 
changes in policy-relevant variables over time. 



Database 

The data reported in this study are drawn primarily from the 1990 NAEP Eighth 
Grade Mathematics Teacher Questionnaire. This questionnaire, along with those 
administered to students and school administrators, was developed under the guidance of 
an Analysis and Use Panel (AUP) composed of policymakers, policy analysts, educational 
researchers, and other individuals who influence policy. The AUP drafted a policy 
information framework that guided the process of developing and selecting items for the 
questionnaires as well as the analysis and reporting of these data (NAEP 1989). Questions 
on the mathematics teacher questionnaires were developed jointly with the NAEP 
Mathematics Item Development Panel and were reviewed by state directors of testing, 
state mathematics supervisors and the National Center for Education Statistics. 

The eighth-grade mathematics teacher questionnaire consists of two parts. The first 
part collects information on the teachers' background, such as academic degrees held, 



Participating jurisdictions are listed in Appendix A. 
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teaching certification, training in mathematics, and availability of instructional resources. 
The second part requests information on each class they taught that included one or more 
students who were assessed. These questions cover instructional practices, such as ability 
grouping, time spent on mathematics instruction, and the use of textbooks, manipulatives, 
small group activities, and mathematics projects, coverage of mathematical topics, and the 
use of calculators and computers. Questions on instructional practices were designed to 
capture u best practice" as well as traditional teaching methods. 

Data from the teacher questionnaire were supplemented with information on the 
characteristics of the states, selected state education policies, and characteristics of students 
assessed in each state. These additional data included cost-adjusted education expenditures 
per pupil for 1989-90, average per capita income for each state in 1989, the percent of 
students receiving free school lunch in each state in 1987, the existence of a state testing 
program in mathematics, the racial/ethnic composition of the assessed students (percent of 
students who are White), the number of literacy items in the students' homes (the percent 
of students reporting four specified reading materials in the home), and the educational 
achievement of the students' parents (the percent of students whose parents had a high 
school diploma or less). 2 

1990 Trial State Assessment Sampling 

Forty jurisdictions (37 states, two territories and the District of Columbia) participated 
in the 1990 NAEP Trial State Assessment. A sample of 100 schools was selected in each 
state, with a sample of 30 students drawn from each school. Schools were stratified based 
on urbanicity, racial/ethnic composition, and median household income. Approximately 
2 ? 500 eighth-grade students were assessed in each jurisdiction. Because the teacher sample 
is based on participating students, they are not necessarily representative of all eighth- 
grade teachers in their state. Therefore, responses to the teacher questionnaire are reported 
using the student as the unit of analysis. Thus, NAEP provides information on the 
instruction received by a representative sample of students, rather than that provided by a 
representative sample of teachers (Mullis et al. 1991). 

Analysis of Data 

Data from the 37 participating states are included in this study. Variables from the 
eighth-grade mathematics teacher questionnaire were correlated with each other and with 
the state characteristics. A subset of teacher background and instructional practice items 
and student demographic variables were also regressed against the state NAEP 
mathematics scores. 



'Sources for each variable are reported in Appendix B. 



Major Findings 



This section describes the variation in instructional practices, resources, teacher 
characteristics, and course-taking in eighth-grade mathematics that are captured in the 
1990 NAEP questionnaires. It groups the variables into five categories: (1) instructional 
practices, (2) topic coverage, (3) resources, (4) course-taking, and (5) teacher experience 
and training. In each category, we describe the range of variation reported across the 37 
participating states and then relate this variation to the characteristics of the states and the 
assessed students. Where appropriate, we also compare the findings of this study to those 
of a much more detailed look at high school mathematics classrooms, Reform Up Close 
(Porter et al., 1993). 3 Although the latter study focuses on high school courses, it includes 
information on instructional practices used in basic mathematics and pre-algebra courses, 
courses which are also taught in the eighth grade. This section begins, however, with a 
description of state characteristics. 

State Characteristics 

Table 1 shows the mean and range of state characteristics used in this study. Cost- 
adjusted expenditures per pupil in 1989-90 ranged from a low of $3,499 in Idaho to a high 
of $6,994 in New York State, with a national average expenditure of $4,952. State average 
per capita income (PCI) in the participating states ranged from a low of $12,345 in West 
Virginia to a high of $24,683 (Connecticut), or with an average PCI of $17,596. Nearly 
one-quarter of the nation's students participated in the federal free school lunch program in 
1987. Louisiana had the largest percentage of students receiving free school lunches in 
1987 — 4o percent — but six other states participating in the Trial State Assessment had free 
school lunch percentages of 30 percent or greater. At the other end of the distribution, 
only 8 percent of New Hampshire's students were in this program. The racial/ethnic 
composition of the states' eighth-grade students also varied considerably, ranging from a 
low of 18 percent White students in Hawaii to a high of 94 percent in New Hampshire. 
Two states in addition to Hawaii — California and Texas — were majority-minority on this 
measure. In five states— Arizona, California, Florida, Hawaii, and New Mexico — 40 
percent or fewer of eighth graders reported they had four or more specified reading 
materials in their homes, in contrast to three states — New Hampshire, Nebraska and North 
Dakota — where 60 to 61 percent of the students reported this many items. The educational 
attainment of eighth-grade parents also differed across states. One-quarter of Idaho's eighth 
graders reported their parents bad a high school diploma or had not completed high 
school. The corresponding percentage was 50 percent in West Virginia. 



3 Reform Up Close is a study of state, district and school policies and practices concerning high school 
mathematics and science instruction. A major component of this project was the collection and analysis 
of detailed data on the content and pedagogy of instruction in four target classes in each of 18 high 
schools. 



Table 2 presents correlations among these state characteristic and their relationship to 
whether the state has a state test in mathematics and the average NAEP proficiency score 
in eighth-grade mathematics. There is a moderate and significant correlation between the 
average level of education spending in the participating states and the income of their 
citizens. Measures of socio-economic status clustered across the states: states with high 
levels of poverty (as measured by the percentage of students receiving free school lunches) 
had more students from homes with fewer reading materials and less well-educated 
parents. Low SES states also had larger minority student populations. There was no 
correlation between state economic and demographic characteristics and the existence of a 
state mathematics test, due in large part to the large number of states administering such 
tests (39 in 1989-90 4 ). Average state performance on the eighth-grade NAEP assessment, 
however, is positively correlated with the level of state education spending, the socio- 
economic status of students in the state, and the racial/ethnic composition of the state's 
students. 



Instructional Practices 

The NAEP mathematics teacher questionnaire asked teachers of assessed students to 
report whether students were assigned to their classes by ability, the amount of time they 
spend each week on mathematics instruction with each class, and how often students in 
their classes do a series of activities for mathematics. 

Ability grouping. Nationally, almost two-thirds of eighth graders have been assigned 
to their mathematics class based on some measure of their ability. At the state level, this 
ranges from a low of 30 percent of students in North Dakota to a high of 93 percent in 
Hawaii and Maryland (see Table 3). As shown in Table 5, the use of ability grouping 
across classes is reported more often in high-income states, states with larger minority 
students populations and in states with statewide mathematics tests. 

Time on mathematics instruction. Teachers were asked to report how much time 
they spend each week on mathematics instruction. Nineteen percent of the nation's eighth 
graders received less than 2.5 hours of mathematics instruction in a week in 1990, while 
30 percent received four or more hours of instruction. Time spent on mathematics 
instruction varied widely across the states. Only 1 1 percent of students in Connecticut and 
New York were in mathematics classes that met less than 2.5 hours a week, compared to 
nearly half of the eighth-grade students in Oklahoma. Only 9 percent of Iowa students 
were in mathematics classes than met more than four hours a week, compared to 57 
percent in Georgia. Student in high-spending and high-wezlth states were less likely to 
have teachers who reported teaching mathematics either 2.5 hours a week or less, or 4 
hours a week or more. 
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4 Coley and Goertz (1990). 
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States with more low SES students were more likely to report above-average amounts 
of mathematics instruction, perhaps reflecting the use of extended instruction for 
compensatory education. This hypothesis is strengthened by the significant negative 
correlation between the amount of instructional time in mathematics and average NAEP 
scores. 

Mathematics activities. Teachers of eighth-grade students were asked how often 
students in their classes performed a variety of activities for mathematics, such as solving 
mathematics problems from textbooks or on worksheets, working in small groups, working 
with manipulatives, taking mathematics tests, and using calculators and computers. In 
1990, teachers made heavy use of problems in textbooks. Nationally, nearly two-thirds of 
eighth graders had teachers who assigned them problems from textbooks almost every day. 
This figure was 55 percent in New Hampshire, the state with the lowest reported use of 
textbook problems. Most eighth-grade students (85 percent) in Alabama and Wyoming 
were assigned problems from their textbooks on a regular basis. 

Many recommendations for the reform of mathematics education advocate the use of 
small-group work, work with concrete materials, and problem-solving in the context of 
projects. For example, the NCTM Standards recommend that classrooms be equipped with 
ample supplies of manipulatives and that students work on mathematical projects and 
prepare reports in mathematics. In 1990, half of the country's eighth graders had teachers 
who reported they used small groups for mathematics at least once a week. Teachers in 
Colorado, Wyoming and Oregon were the most likely to use this approach; teachers in 
Rhode Island the least likely. 

Teachers of eighth-grade mathematics were less likely to use manipulatives or to 
assign their students reports or projects in mathematics. About one-third of eighth-grade 
students in Colorado, California, Montana and Oregon had teachers who used rulers, 
counting blocks and geometric shapes in their classrooms. The national average was 22 
percent. Forty-three percent of eighth-grade students nationally were never assigned reports 
or projects. This percentage rose to over 60 percent in Wyoming, Pennsylvania, Delaware 
and Rhode Island. Over 60 percent of eighth-grade students in North Carolina, Georgia, 
Kentucky, New Jersey and Virginia, however, were assigned an occasional paper or report 
in mathematics. 

These figures are supported by findings from Porter and others (1993). They found 
that high school teachers in their study allocated virtually no instructional time in basic 
mathematics, pre-algebra or Algebra 1 classes to students' writing (zero to one percent) or 
to lab/field work (zero to 3 percent). Less than 10 percent of instructional time was spent 
on student presentations or demonstrations in these classes. 

Neither calculators nor computers were a fixture in eighth-grade classrooms in most 
states in 1990. Nationally, only 27 percent of eighth-grade students had teachers who used 
calculators to teach mathematics at least several times a week. Students in New York State 
were the least likely to have exposure to calculators, while calculators were used the most 
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extensively in Montana. More than half of all eighth graders were in classrooms where the 
computer was never used to teach mathematics. Students in Montana again had the 
greatest exposure to computer use in mathematics, while those in Louisiana had the least 
exposure. Frequent use of calculators and use of computers were positively correlated with 
the state NAEP scorf\ however. 

Finaliy, teachers were asked how often they give teacher-generated mathematics tests. 
Nationally, only 39 percent of eighth-grade students have teachers who test them less than 
once a week. Students are tested most often in Louisiana, and least often in Oregon. 

At the state level, one begins to see instructional practices fall into two groups: 
"traditional" (heavy reliance on textbooks and little use of other activities) and "non- 
traditional" (less reliance on textbooks and a greater use of manipulatives, small groups, 
reports, calculators and computers). As shown in Table 4, for example, students in states 
where teachers assign mathematics problems from textbooks on a daily basis are less 
likely to work in small groups, work with manipulatives, write reports or do mathematics 
projects, or use calculators on a regular basis. Their teachers are also more likely to place 
a heavy emphasis on the teaching of numbers and operations (see Topic Coverage below). 
Students in states where teachers use small groups on a regular basis are also more likely 
to work with manipulatives and calculators. Their teachers place less emphasis on the 
teaching of numbers and operations, and test them less often. These preliminary patterns 
are worth exploring using a national, rather than state-level, teacher database, and at the 
fourth-, as well as eighth-grade level. 

Four of the instructional practices are correlated with state characteristics (Table 5). 
The only activity related to level of spending on education is the use of computers. 
Teachers in high-spending states are less likely to report they do not use computers at all 
to teach mathematics. This relationship is not surprising, as the availability of computers in 
eighth-grade classrooms is also related to state spending levels. Students in states with 
higher concentrations of low SES students are more likely to be assigned problems from 
textbooks and tested more often than students in states with more advantaged student 
populations. Students in states with less advantaged students are also less likely to use 
calculators and computers on a regular basis. A small, but significant, positive correlation 
also exists between the existence of a state mathematics test and the frequency with which 
students are tested. 



Topic Coverage 

Eighth-grade mathematics teachers were asked to report the emphasis they placed on 
numbers and operations (e.g., whole number operations, common fractions, decimal 
fractions, ratio or proportion, and percent), measurement, geometry, data analysis, statistics 
and probability, and algebra. Nearly half of the nation's eighth-grade students were in 
mathematics classes that placed a heavy emphasis on numbers and operations (49 percent) 
and algebra and functions (46 percent). Less emphasis was placed on geometry and data 



8 



analysis, statistics and probability. The content of eighth-grade mathematics classes varied 
by state. Teachers in Texas, for example, placed greater emphasis on numbers and 
operations, while those in Louisiana stressed the teaching of algebra. 

These reports are similar to the findings of Reform Up Close. That study found that 
basic mathematics courses in high school spent more than half of their time on arithmetic, 
and paid little attention to geometry, statistics and probability (Porter et al. 1993). The two 
studies differ on emphasis on algebra, however. Only 10 percent of instructional time in 
nigh school basic mathematics courses in the Reform Up Close study was devoted to 
algebra, while algebra was given heavy emphasis in half of eighth-grade mathematics 
classes. There are three possible explanations for this difference. First, about 35 percent of 
eighth-grade students are enrolled in pre-algebra and algebra classes and the instruction 
they receive is averaged in with the instruction provided in basic eighth-grade mathematics 
classes. Second, basic eighth-grade mathematics courses, which prepare students for: high 
school mathematics, may contain more algebra than basic mathematics classes in high 
school, which are more remedial in focus. Third, the Reform Up Close study collected 
much more detailed instructional information from teacher logs and questionnaires. Th°Jr 
definition of what constitutes algebra instruction may be much narrower than that used by 
teachers in the NAEP questionnaire. 

Teachers in the NAEP sample were also asked how much emphasis they placed on 
four mathematics skills: learning mathematics facts and concepts, learning skills and 
procedures needed tc solve ro-vie problems, developing reasoning and analytic ability to 
solve unique problems, and learning how to communicate ideas in mathematics effectively. 
The most emphasis was placed on learning skills to solve routine problems, followed by 
emphasis on learning facts and concepts. Less attention was paid to learning how to 
communicate ideas in mathematics, although emphasis varied across the states on all four 
of these skill areas. 

Greater emphasis was placed on numbers and operations and learning facts and 
concepts in low-income states and states with low student SES, while greater emphasis 
was placed on geometry and developing reasoning and analytic ability in high-income 
states. There was no correlation between the con'.ent of eighth-grade mathematics courses 
and the existence of a state mathematics test, v ith the exception of a moderate, positive 
relationship with learning how to communi -ate ideas in mathematics. States that placed a 
greater emphasis on the teaching of numbers anu operations and facts and concepts had 
relatively lower NAEP scores, as did those that emphasized the development of reasoning 
and communication skills. States with a strong emphasis on algebra had higher proficiency 
scores, however. 



Resources 

Teachers were asked if their school system provides them with all, most, some or 
none of the instructional materials and other resources they need to teach their classes. 
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Nearly one-third of the nation's eighth-grade students have mathematics teachers who 
report they receive none or only some of the resources they need (Table 6). This 
percentage ranges from a low of 14 percent in Iowa to a high of 58 percent in Louisiana. 
Teachers in low-spending and low-income states, in states with low-SES students, and in 
states with large minority student populations were more likely to report a shortage of 
resources than were teachers in more advantaged states (Table 7). We also found a strong, 
positive correlation between state average NAEP mathematics scores and teacher reports of 
the sufficiency of instructional resources. 

Eighth-grade mathematics teachers were also asked about the availability of 
calculators and computers in their schools. Fifty-six percent of eighth-grade students have 
access to school-owned calculators in their mathematics class, ranging from a low of 29 
percent of students in Louisiana to a high of 89 percent of students in California. Only 22 
percent of eighth-grade students have a computer available in their mathematics classroom; 
another 28 percent of stuaents have no access to a computer for use in their mathematics 
class. Again, the percentages vary across states. Only 7 percent of eighth-grade students in 
Minnesota have no access to a computer for mathematics class, while more than half (57 
percent) of students in Louisiana are without computers. Access to both calculators and 
computers in the school is positively related to levels of education spending, income and 
student SES in the states. In addition, computers are less available in states with large 
percentages of minority students. 



Course-taking 

Eighth-grade students were asked whether they were enrolled in eighth-grade 
mathematics, pre-algebra or algebra. Fifteen percent of the students reported they were 
taking algebra. This percentage ranged from a low of 8 percent in North Dakota to a high 
of 27 percent in Maryland. Eighth-grade students were more likely to take algebra in high- 
spending, high-income states. On an individual level, eighth-grade students enrolled in 
algebra had, on average, higher proficiency scores than those students enrolled in other 
mathematics courses (Mullis et al. 1991). This relationship did not hold on the state level, 
however. There was no correlation between the percent of students enrolled in algebra and 
state average mathematics proficiency scores. 

Teacher Experience and Training 

NAEP collected information on the training and experience of the mathematics 
teachers of assessed eighth-grade students. While research has been inconsistent about the 
relationship between years of teaching experience and degree attainment and student 
achievement, three variables — years of teaching mathematics, majoring in mathematics and 
taking a broad range of mathematics courses — are positively related to eighth-grade 
mathematics achievement at both the individual level (Mullis et al. 1991) and at the state 
level. 



Eighth-grade students nationally are taught by staff who have taught mathematics an 
average of 14 years. Average mathematics teaching experience ranges from a low of 11 
years in six states (Arizona, Arkansas, Idaho, New Mexico, Texas, and West Virginia) to 
17 years in three states (Iowa, Minnesota, Wisconsin). Forty-three percent of students are 
taught mathematics by teachers who majored in mathematics in college. At the state level, 
this ranges from a low of 15 percent in Arizona to a high of 88 percent in Minnesota. As 
a measure of breadth of training in mathematics, NAEP reports that 52 percent of students 
had eighth-grade mathematics teachers who took college courses in at least six of seven 
mathematics content areas. 5 Again, this percentage ranged from a low of 25 percent in 
Kentucky to a high of 85 percent in Minnesota. Teachers who were mathematics majors 
were more likely to report this breadth of training than those who majored in other 
subjects (correlation of 0.83). 

It is also important to track the amount of in-service training that mathematics 
teachers receive, especially given how long it has been since most teachers completed their 
formal education. Nearly 40 percent of students were taught by teachers who had 
completed 16 hours or more of in-service training in mathematics or the teaching of 
mathematics the previous year. Although this is not a lot of training (only two days), only 
16 percent of eighth-grade students in Indiana had teachers with this level of in-service 
participation. At the high end, 69 percent of students in New Hampshire had teachers with 
this level of recent training. Although the amount of in-service training is not directly 
correlated with mathematics achievement, it is correlated with instructional practices when 
data are aggregaced to the state level (see Table 4). Students who had eighth-grade 
mathematics teachers who received 16 or more hours of in-service education in 
mathematics or mathematics teaching methods were more likely to be instructed by 
teachers who used non-traditional instructional practices, such as small groups, 
manipulatives, report writing and calculators on a regular basis. They were also Lss likely 
to be assigned problems from textbooks or have a heavy emphasis placed on the teaching 
of numbers and operations. 

The only teacher training and experience variable related to state education 
expenditure and state income was the number of years that teachers taught mathematics. 
However, the demographics of the states' students was positively related to the training 
and experience of their teachers. States with high percentages of students receiving free 
lunch and with few reading materials in the home were less likely to have teachers trained 
in mathematics or who had taught the subject for a long period of time. Students in states 
with high percentages of minority students were also less likely to be taught by well - 
trained teachers. There was no relationship, however, between the socio-economic 
characteristics of the states and the percentage of teachers who spent 16 or more hours in 
mathematics-delated in-service training. 



5 The seven areas are number systems and numeration, geometry, probability and statistics, abstract or 
linear algebra, calculus, computer science, and computer programming. 
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Relationship of Instructional Practices and Teacher Characteristics with 
State NAEP Mathematics Scores 



We conducted a series of multiple regressions to examine the relationship between 
four sets of instructional practices and teacher characteristics with state NAEP mathematics 
scores: (1) resources, (2) instructional practices, (3) topic coverage, and (4) teacher 
characteristics. 

Resources. The first block included measures of resources available to eighth-grade 
mathematics classrooms: (1) teachers get some or none of the resources they need; (2) 
students have access to school calculators; and (3) computers are available for students to 
use. Taken together, the three variables explain 57 percent of the variance in state NAEP 
mathematics scores, but only the first variable is statistically significant. 

Instructional practices. This block included six measures of instructional practice: 
(1) teachers use calculators at least several times a week; (2) students work in small 
groups at least once a week; (3) students are assigned 'o mathematics class based on 
ability; (4) students use computers at le&*t once a week, (5) students do problems from 
textbooks almost every day; and (6) students never write reports or do mathematics 
projects. This block of variables explain 56 percent of the variance in state NAEP scores, 
but most of this is explained by the first two variables. In addition, holding the use of 
calculators constant, the relationship between use of small groups and tested achievement 
becomes negative. 6 

Topic coverage. In the third block, we regressed the emphasis placed on the five 
topics and four mathematics skills addressed by the NCTM standards. This block of 
variables explains 62 percent of the variance of NAEP state mathematics scores, with most 
of the variance explained by heavy emphasis on learning concepts, algebra, learning to 
communicate ideas, and numbers and operations. 

Teacher characteristics. This block, which included (1) years of teaching 
mathematics, (2) percent of teachers with only an undeigraduate degree, (3) percent of 
teachers with an undergraduate mathematics major, (4) percent of teachers certified in 
mathematics, and (5) percent of teachers who spent 16 hours or more on mathematics in- 
service in the prior year, accounted for 40 percent of the difference in state mathematics 
scores. Only the first two measures were statistically significant, and accounted for most of 
the variance. 

If is difficult, however, to factor out the influence of student SES on these relation- 
ships. As we have seen, there is not only a strong correlation between the percent of low- 
SES students in a state and NAEP mathematics scores, but a strong correlation between 



6 The correlation between the use of small groups and mathematics achievement is small and 
insignificant but is positive. 
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the SES of a state and many of those educational opportunities that seem to explain 
differences in state mathematics scores. When one can explain more than 80 percent of the 
variance in state NAEP scores with three student demographic variables — percent of 
eighth-grade students living in a two-parent family, the percent of students whose parents 
have formal education beyond high school, and the percent of students in a state who are 
White — what is left to be explained by differences in instructional practices? This is the 
"North Dakota phenomenon," where high NAEP scores can be explained in large part by 
the homogeneous and stable student population in the state. 

One way to explore the explanatory power of different categories of variables is 
through blockwise regression analysis. The first block of variables entered in the equation 
was "instructional practices." As discussed above, this block accounted for 56 percent of 
the variance in NAEP scores across states. The second block we entered was "resources." 
The availability of resources to teachers and their students explains an additional 24 
percent of the variance (see Table 8). Topic coverage accounts for another 9 percent of the 
variance, when this block of variables is entered into the regression following instructional 
practices and level of resources. The block of variables measuring the training and 
experience of teachers adds only another 2 percent to the explanation. Taken together, 
however, "these four blocks account for 91 percent of the difference in average eighth- 
grade NAEP mathematics scores across the states. The student demographics block does 
not have much independent explanatoi;' power — only 6 percent — but much of the 
predictive power of the other blocks is correlated with student demographics and the 
socio-economic characteristics of the states. 
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Policy Implications and Plans for the Year 2 Study 



The data presented in this report can be used by policymakers for three purposes: (1) 
to determine the extent to which instructional practices recommended by mathematics 
reformers (such as the NCTM) have been enacted across the states; (2) to examine the 
degree to which opportunities to learn the "new mathematics" are related to economic and 
socio-economic characteristics of states; and (3) to track changes in eighth-grade 
mathematics instructional practices over time. 

There are limitations, however, to the use of data aggregated at the state-level data for 
policy development. While the kinds of analyses presented here allow us to look generally 
at differences across states, this aggregation may obscure differences across schools within 
states. One needs to use more sophisticated statistical techniques, such as multi-level linear 
models, to examine variation in instructional practices within states and the interaction of 
within and across state variations. This approach would enable one to determine, for 
example, whether and to what extent disadvantaged students living in disadvantaged states 
are doubly penalized in their opportunities to learn mathematics. Hierarchical linear 
models could also be used in conjunction with the NAEP database to identify differences 
among teachers in the same school (cf M Raudenbush, Rowan and Cheong 1993). For 
example, the use of calculators may vary across classes within schools because of 
differences in course content (calculators may be used more often in advanced mathe- 
matics classes) or differences in teachers' individual practice. The National Center for 
Education Statistics has issued RFPs for small-scale analyses of the NAEP assessment and 
questionnaire data. It is hoped that some of these studies will begin to disentangle some of 
these multi-layered relationships. 

The Enactment of New Instructional Practices in Mathematics 

This study found that a majority of eighth-grade students receive mathematics 
instruction in traditional classrooms. They are assigned to their classrooms based on their 
ability, they receive between 2.5 and 4 hours of mathematics instruction a week by 
teachers who place heavy emphasis on numbers and operation, and facts and concepts, but 
pay limited attention to geometry and statistics and probability. The students are more 
likely to do problems from textbooks than to do reports or projects on mathematics. Few 
students use calculators in mathematics class on a regular basis, and more than half of the 
students never use ? computer. 

At the state level, however, one begins to see instructional practices fall into two 
groups: "traditional" (heavy reliance on textbooks and little use of other activities and a 
heavy emphasis on the teaching of number and operations) and "non-traditional" (less 
reliance on textbook problems, a greater use of manipulatives, small groups, reports, 



calculators and computers, and less emphasis on numbers and operations). In addition, it 
appears that eighth-grade mathematics teachers that have participated in at least 16 hours 
of in-service training in mathematics and/or the teaching of mathematics, in the last year 
are more likely to report using non-traditional instructional practices. Although these 
patterns are based on aggregated data and fairly simple statistical methods, this is the first 
study to examine relationships ai long instructional practices. To date, reports published 
using the NAEP data have focused only on the relationship of discrete instructional 
practices and NAEP test score data (cf., Mullis et al. 1991; CCSSO 1993). Relationships 
among instructional practices are worth exploring further using a national, rather than a 
state-level, data base, and at the fourth, as well as eighth-grade level. 

Differences in Instructional Practices Across States 

The 1990 NAEP Trial State Assessment shows that there is wide variation in 
instructional practices across the states, and that students' experiences in their classrooms 
are often related to the characteristics of their states. For example, students living in high- 
spending states are more likely to have access to calculators and computers, have teachers 
with adequate instructional resources, and have more experienced teachers. Students in 
high-spending states are also more likely to take algebra in eighth grade. Students in states 
with a higher proportion of disadvantaged students are less likely to have access to 
calculators and computers, more likely to be taught using traditional instructional practices 
(e.g., do problems from a textbook on a daily basis, have heavy emphasis placed on 
numbers and operations and ff ?ts and concepts, and be tested at least weekly), and less 
likely to have experienced teachers or teachers with considerable training in mathematics 
(e.g., an undergraduate major in mathematics and/or course work in several areas of 
mathematics). They are also less likely to be enrolled in algebra in eighth grade. 

When groups of variables are regressed against the state NAEP proficiency score, it 
appears that resources available to teachers and certain instructional practices, such as use 
of calculators and emphasis on different mathematics topics and skills explain considerable 
variance in state NAEP scores. It is difficult, however, to factor out the influence of 
student SES on these relationships. For example, one can explain more than 80 percent of 
the variance in state NAEP scores with three student demographic variables: (1) percent of 
eighth-grade students in the state living in a two-parent family, (2) the percent of students 
whose parents have formal education beyond high school, and (3) the percent of students 
in a state who are White. This is the "North Dakota phenomenon," where high NAEP 
scores can be explained in large part by the homogeneous and stable student population in 
the state. While student demographics do not have much independent explanatory power, 
they are highly correlated with other blocks of factors that explain much of the variation in 
test scores across the states. 



21 

16 



Tracking Changes in instructional Practices 



This study provides a baseline for tracking changes in instructional practices in 
eighth-grade mathematics. The NAEP 1992 Mathematics Report Card for the Nation and 
States provides detailed information on student tested achievement, but none on teacher or 
school policies and practices. The second phase of this project, scheduled for 1994, will 
examine changes in eighth-grade mathematics instructional practices and NAEP test scores 
between 1990 and 1992 in those states that participated in the State Trial Assessment in 
both years. Using data on state characteristics, we will identify the characteristics of the 
states that have changed the most, and the impact of these changes on equity in the 
opportunity to learn mathematics and tested achievement. Have instructional practices 
changed as teachers, schools and states respond to the NCTM standards? Has the 
relationship between instructional practices and state wealth, expenditures and SES 
lessened? What are the implications of these changes for students of different social, 
economic and racial backgrounds? 
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Table 1 

Distribution of State Characteristics across the 37 Participating States 





National 
Average 
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State 
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West Virginia 
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8% 

New Hampshire 


46% 
Louisiana 
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18% 
Hawaii 


94% 

New Hampshire 
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Materials in the Home 


48% 


35% 
Hawaii 


61% 
New Hampshire 
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High School Education or Less 
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Idaho 


50% 
West Virginia 
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Table 3 

Distribution of Teacher Responses across the 37 Participating States 





National 
Average 


Lowest 
a tale 


Highest 


Use Ability Grouping 


63% 


30% 
North Dakota 


93% 

Hawaii, Maryland 


Less than 2.5 Hours of 
Math Instruction 


19% 


n% 

Connecticut, 
New York 


46% 
UKianoma 


4.0 or More Hours of 
Math Instruction 


jU no 


y /o 
Iowa 


J / tO 

Georgia 


Textbook Problems Daily 


62% 


55% 
New Hampshire 


OC/W 

OJ70 

Alabama, West Virginia 


Small Groups Weekly 


50% 


27% 
Rhode Island 


10% 
Oregon, Wyoming 


Use Manipulatives 


22% 


11% 

Indiana, Pennsylvania 


37% 
Montana 


No Reports/Projects 


43% 


35% 
North Carolina 


63% 
Rhode Island 


Teachers Tests Rarely 


3y70 


iZ/o 
Louisiana 


\J\J /o 
Oregon 


Use Calculators Often 


ll7o 


0 /o 
New York 


\J\J /o 
Montana 


iNever use v^umpuier 


■J /u 


37%' 
Montana 


81% 
Louisiana 


Emphasis on Numbers and 
Operation 


49% 


34% 
Oregon 


61% 
Texas 


Emphasis on Geometry 


28%- 


14% 
Idaho 


40% 
New York 


Emphasis on Data Analysis, 

Ouzildili'df aim rnjuiciiio 


14% 


4% 
Indiana 


24% 
Georgia 


Emphasis on Algebra and 
Functions 


46% 


29% 

naWuii 


59% 
Louisiana 


Emphasis on Facts and 
Concepts 


55% ■ 


47% 
Minnesota 


72% 
Kentucky 


Emphasis on Problem-Solving 
Skills 


67% 


56% 
Montana, Oregon 


75% 
Pennsylvania, 
West Virginia 


Emphasis on Reasoning Ability 


45% 


33% 
North Dakota 


53% 
Maryland 


Emphasis on Communicating 
Ideas in Math 


37% 


24% 
Wisconsin 


52% 
Georgia 
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Table 6 

Distribution of Teacher Responses across the 37 Participating Stages 





National 
Average 


Lowest 
State 


Highest 
State 


Have some or no resources 


31% 


14% 
Iowa 


58% 

Louisiana 


Access to calculators 


56% 


29% 
Louisiana 


89% 
Connecticut 


No access to computers 


. 28% 


7% 
Minnesota 


57% 
Louisiana 


Take algebra in 8th grade 


15% 


8% 
North Dakota 


27% 
Maryland 


Years baching math 


14 


11 

Arizona, Arkansas, Idaho, 
New Mexico, Texas, West 
Virginia 


17 
Iowa, 
Minnesota, 
Wisconsin 


Have math certification 


84% 


41% 
Arizona 


98% 
Minnesota 


Undergrad major in math 


43% 


15% 
Arizona 


88% 
Minnesota 


Number of math areas 
covered 


52% 


25% 
Kentucky 


85% 
Minnesota 


16 or more hours in- 
service in math in last year 
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16% 
Indiana 


69% 
New Hampshire 
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Table 8 

Predictors of Average State NAEP Scores, 8th Grade Mathematics 



Block R: 

Block 1: Instructional Practices 0.56 

Block 2: Resources 0.80 

Block 3: Topic Coverage 0.89 

Block 4: Teacher Characteristics 0.91 

Block 5: Student Demographics 0.97 

ONLY Student Demographics 0.81 
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Appendix A 



States Participating in the 1990 NAEP Trial State Assessment 



Alabama 


Idaho 


Nebraska 


Oregon 


Arizona 


Illinois 


New Hampshire 


Pennsylvania 


Arkansas 


Indiana 


New Jersey 


Rhode Island 


California 


Iowa 


New Mexico 


Texas 


Colorado 


Kentucky 


New York 


Virginia 


Connecticut 


Louisiana 


North Carolina 


West Virginia 


Delaware 


Maryland 


North Dakota 


Wisconsin 


Florida 


Michigan 


Ohio 


Wyoming 


Georgia 


Minnesota 


Oklahoma 




Hawaii 


Montana 







Appendix B 



Sources of Supplementary Data 

1. Cost-adjusted education expenditures per pupil for 1989-90: Total current expenditures 
for public elementary and secondary day schools per pupil in average daily attendance, 
from National Education Association, Estimates of School Statistics, 1990-91, adjusted by 
a cost index reported in Nelson (1991). Adjusted figures are taken from Barton, Coley and 
Goertz (1991). 

2. Average per capita income for each state in 1989: U. S. Department of Commerce 
Bureau of Economic Analysis, Commerce News, August 1990, as reported in Mullis, 
Dossey, Owen and Phillips (1991), Table A.4. 

3. Percent of students receiving free school lunch in each state in 1987. Calculated from 
data provided by U.S. Department of Agriculture, Food and Nutrition Service, 1987, and 
Statistical Abstract of the United States: 1987, as reported in Mullis, Dossey, Owen and* 
Phillips (1991), Table A.4. 

4. Racial/ethnic composition of the assessed students: 1990 NAEP eighth-grade student 
questionnaire. 

5. Number of literacy items in the students 1 homes: 1990 NAEP eighth-grade student 
questionnaire. 

6. Educational achievement of the students' parents: 1990 NAEP eighth-grade student 
questionnaire. 

7. State testing mandate: Coley and Goertz (1990). 
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